{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Optional Lab: Model Representation\n",
    "\n",
    "<figure>\n",
    " <img src=\"../work/images/C1_W1_L3_S1_Lecture_b.png\"   style=\"width:600px;height:200px;\">\n",
    "</figure>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Goals\n",
    "In this lab you will:\n",
    "- Learn to implement the model $f_{w,b}$ for linear regression with one variable"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Notation\n",
    "Here is a summary of some of the notation you will encounter.  \n",
    "\n",
    "|General <img width=70/> <br />  Notation  <img width=70/> | Description<img width=350/>| Python (if applicable) |\n",
    "|: ------------|: ------------------------------------------------------------||\n",
    "| $a$ | scalar, non bold                                                      ||\n",
    "| $\\mathbf{a}$ | vector, bold                                                      ||\n",
    "| **Regression** |         |    |     |\n",
    "|  $\\mathbf{x}$ | Training Example feature values (in this lab - Size (1000 sqft))  | `x_train` |   \n",
    "|  $\\mathbf{y}$  | Training Example  targets (in this lab Price (1000s of dollars)).  | `y_train` \n",
    "|  $x^{(i)}$, $y^{(i)}$ | $i_{th}$Training Example | `x_i`, `y_i`|\n",
    "| m | Number of training examples | `m`|\n",
    "|  $w$  |  parameter: weight,                                 | `w`    |\n",
    "|  $b$           |  parameter: bias                                           | `b`    |     \n",
    "| $f_{w,b}(x^{(i)})$ | The result of the model evaluation at $x^{(i)}$ parameterized by $w,b$: $f_{w,b}(x^{(i)}) = wx^{(i)}+b$  | `f_wb` | \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Tools\n",
    "In this lab you will make use of: \n",
    "- NumPy, a popular library for scientific computing\n",
    "- Matplotlib, a popular library for plotting data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "plt.style.use('./deeplearning.mplstyle')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Problem Statement\n",
    "<img align=\"left\" src=\"../work/images/C1_W1_L3_S1_trainingdata.png\"    style=\" width:380px; padding: 10px;  \" /> \n",
    "\n",
    "As in the lecture, you will use the motivating example of housing price prediction.  \n",
    "This lab will use a simple data set with only two data points - a house with 1000 square feet(sqft) sold for \\\\$300,000 and a house with 2000 square feet sold for \\\\$500,000. These two points will constitute our *data or training set*. In this lab, the units of size are 1000 sqft and the units of price are 1000s of dollars.\n",
    "\n",
    "| Size (1000 sqft)     | Price (1000s of dollars) |\n",
    "| -------------------| ------------------------ |\n",
    "| 1.0               | 300                      |\n",
    "| 2.0               | 500                      |\n",
    "\n",
    "You would like to fit a linear regression model (shown above as the blue straight line) through these two points, so you can then predict price for other houses - say, a house with 1200 sqft.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Please run the following code cell to create your `x_train` and `y_train` variables. The data is stored in one-dimensional NumPy arrays."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# x_train is the input variable (size in 1000 square feet)\n",
    "# y_train is the target (price in 1000s of dollars)\n",
    "x_train = np.array([1.0, 2.0])\n",
    "y_train = np.array([300.0, 500.0])\n",
    "print(f\"x_train = {x_train}\")\n",
    "print(f\"y_train = {y_train}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    ">**Note**: The course will frequently utilize the python 'f-string' output formatting described [here](https://docs.python.org/3/tutorial/inputoutput.html) when printing. The content between the curly braces is evaluated when producing the output."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Number of training examples `m`\n",
    "You will use `m` to denote the number of training examples. Numpy arrays have a `.shape` parameter. `x_train.shape` returns a python tuple with an entry for each dimension. `x_train.shape[0]` is the length of the array and number of examples as shown below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# m is the number of training examples\n",
    "print(f\"x_train.shape: {x_train.shape}\")\n",
    "m = x_train.shape[0]\n",
    "print(f\"Number of training examples is: {m}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "One can also use the Python `len()` function as shown below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# m is the number of training examples\n",
    "m = len(x_train)\n",
    "print(f\"Number of training examples is: {m}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Training example `x_i, y_i`\n",
    "\n",
    "You will use (x$^{(i)}$, y$^{(i)}$) to denote the $i^{th}$ training example. Since Python is zero indexed, (x$^{(0)}$, y$^{(0)}$) is (1.0, 300.0) and (x$^{(1)}$, y$^{(1)}$) is (2.0, 500.0). \n",
    "\n",
    "To access a value in a Numpy array, one indexes the array with the desired offset. For example the syntax to access location zero of `x_train` is `x_train[0]`.\n",
    "Run the next code block below to get the $i^{th}$ training example."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "i = 0 # Change this to 1 to see (x^1, y^1)\n",
    "\n",
    "x_i = x_train[i]\n",
    "y_i = y_train[i]\n",
    "print(f\"(x^({i}), y^({i})) = ({x_i}, {y_i})\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plotting the data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can plot these two points using the `scatter()` function in the `matplotlib` library, as shown in the cell below. \n",
    "- The function arguments `marker` and `c` show the points as red crosses (the default is blue dots).\n",
    "\n",
    "You can use other functions in the `matplotlib` library to set the title and labels to display"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Plot the data points\n",
    "plt.scatter(x_train, y_train, marker='x', c='r')\n",
    "# Set the title\n",
    "plt.title(\"Housing Prices\")\n",
    "# Set the y-axis label\n",
    "plt.ylabel('Price (in 1000s of dollars)')\n",
    "# Set the x-axis label\n",
    "plt.xlabel('Size (1000 sqft)')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Model function\n",
    "\n",
    "<img align=\"left\" src=\"../work/images/C1_W1_L3_S1_model.png\"     style=\" width:380px; padding: 10px; \" > As described in lecture, the model function for linear regression (which is a function that maps from `x` to `y`) is represented as \n",
    "\n",
    "$$ f_{w,b}(x^{(i)}) = wx^{(i)} + b \\tag{1}$$\n",
    "\n",
    "The formula above is how you can represent straight lines - different values of $w$ and $b$ give you different straight lines on the plot. <br/> <br/> <br/> <br/> <br/> \n",
    "\n",
    "Let's try to get a better intuition for this through the code blocks below. Let's start with $w = 100$ and $b = 100$. \n",
    "\n",
    "**Note: You can come back to this cell to adjust the model's w and b parameters**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "w = 100\n",
    "b = 100\n",
    "print(f\"w: {w}\")\n",
    "print(f\"b: {b}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, let's compute the value of $f_{w,b}(x^{(i)})$ for your two data points. You can explicitly write this out for each data point as - \n",
    "\n",
    "for $x^{(0)}$, `f_wb = w * x[0] + b`\n",
    "\n",
    "for $x^{(1)}$, `f_wb = w * x[1] + b`\n",
    "\n",
    "For a large number of data points, this can get unwieldy and repetitive. So instead, you can calculate the function output in a `for` loop as shown in the `compute_model_output` function below.\n",
    "> **Note**: The argument description `(ndarray (m,))` describes a Numpy n-dimensional array of shape (m,). `(scalar)` describes an argument without dimensions, just a magnitude.  \n",
    "> **Note**: `np.zero(n)` will return a one-dimensional numpy array with $n$ entries   \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def compute_model_output(x, w, b):\n",
    "    \"\"\"\n",
    "    Computes the prediction of a linear model\n",
    "    Args:\n",
    "      x (ndarray (m,)): Data, m examples \n",
    "      w,b (scalar)    : model parameters  \n",
    "    Returns\n",
    "      y (ndarray (m,)): target values\n",
    "    \"\"\"\n",
    "    m = x.shape[0]\n",
    "    f_wb = np.zeros(m)\n",
    "    for i in range(m):\n",
    "        f_wb[i] = w * x[i] + b\n",
    "        \n",
    "    return f_wb"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's call the `compute_model_output` function and plot the output.."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "tmp_f_wb = compute_model_output(x_train, w, b,)\n",
    "\n",
    "# Plot our model prediction\n",
    "plt.plot(x_train, tmp_f_wb, c='b',label='Our Prediction')\n",
    "\n",
    "# Plot the data points\n",
    "plt.scatter(x_train, y_train, marker='x', c='r',label='Actual Values')\n",
    "\n",
    "# Set the title\n",
    "plt.title(\"Housing Prices\")\n",
    "# Set the y-axis label\n",
    "plt.ylabel('Price (in 1000s of dollars)')\n",
    "# Set the x-axis label\n",
    "plt.xlabel('Size (1000 sqft)')\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can see, setting $w = 100$ and $b = 100$ does *not* result in a line that fits our data. \n",
    "\n",
    "### Challenge\n",
    "Try experimenting with different values of $w$ and $b$. What should the values be for a line that fits our data?\n",
    "\n",
    "#### Tip:\n",
    "You can use your mouse to click on the triangle to the left of the green \"Hints\" below to reveal some hints for choosing b and w."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<details>\n",
    "<summary>\n",
    "    <font size='3', color='darkgreen'><b>Hints</b></font>\n",
    "</summary>\n",
    "    <p>\n",
    "    <ul>\n",
    "        <li>Try $w = 200$ and $b = 100$ </li>\n",
    "    </ul>\n",
    "    </p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Prediction\n",
    "Now that we have a model, we can use it to make our original prediction. Let's predict the price of a house with 1200 sqft. Since the units of $x$ are in 1000's of sqft, $x$ is 1.2.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "w = 200                         \n",
    "b = 100    \n",
    "x_i = 1.2\n",
    "cost_1200sqft = w * x_i + b    \n",
    "\n",
    "print(f\"${cost_1200sqft:.0f} thousand dollars\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Congratulations!\n",
    "In this lab you have learned:\n",
    " - Linear regression builds a model which establishes a relationship between features and targets\n",
    "     - In the example above, the feature was house size and the target was house price\n",
    "     - for simple linear regression, the model has two parameters $w$ and $b$ whose values are 'fit' using *training data*.\n",
    "     - once a model's parameters have been determined, the model can be used to make predictions on novel data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  },
  "toc-autonumbering": false
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
